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Sub -sample the Content to improve Average Color 
Extraction for Controlling Ambient Lighting Systems 

Current systems for automatic triggering of lights based on the content process each and 
every frame. Such an approach has two obvious disadvantages: (a) the computational 
load needed to do such processing becomes very large and thus leads to synchronization 
problems. In other words there is a delay between what is displayed on the screen and the 
triggered light effects and (b) processing of each and every frame leads to undesirable 
flickering due to subtle color changes in the content. These disadvantages were addressed 
by using subsampling, i.e. processing only one in N frames. So only for one in N frames, 
the actual LED colors are calculated and for the mtermediate frames, the colors are 
linearly interpolated. This reduces flickering a lot and at the same time it makes the 
algorithm an order of N times faster. 



The general steps that need to be followed to realize the invention are as follows: 

1) Acquire a video signal and decode the video signal into a set of frames 

2) Reduce frames by subsampling 

3) Extract color information from the content (frames) around the boundary 

4) Transform the color information of the content from the RGB space onto the color 
space of the LEDs and the displays color space. 

5) Use linear mterpolation to compute the color information for intermediate frames 

6) Transmit the color output to the LED units so as to trigger them. 

Steps (1) and (6) are straightforward and are not fturttier discussed below. 
The current setup for an ambient lighting system is as shown below: 



Sub-light 




Figure 1. Sctap of Li^t Units 
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In the above figure there are 11 independently controllable LED lighting units. There are 
four light speakers, one unit under the couch ('Sub-light') and 6 LED units on the center- 
light. The center-light is a little bit special, in the way it is set-up - it has 6 independently 
controlled, light units behind the four sides of the Flat TV and is shown below: 
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Figure 2: Setiq> of light units around the TV 



In the above figure LI to L6 refer to the light units around TV. The figure also shows a 
single frame of the content displayed on the TV. Each light unit located at the back of the 
TV is triggered by extracting the average color information from each region - Rl to R6. 
Each region has a width of 100 pixels. As an example if the size of the frame were 
720x576 pixels, then the size of Rl. R2, R4 and R5 would be 360x100 pixels. Smularly 
size of R3 and R6 would be 100x376 pixels. 

Since the video signal is decoded into a set of frames (25 frames per second) in the RGB 
color space, the resulting image size would be 720x576x3 which is a 3D matnx where 
each 2D matrix of size 720x576 corresponds to each one of the Red, Green and Blue 
channels. 

Before the color information could be extracted from the frames, first perform 
subsampling. i.e., process 1 mN frames. The optimal value has been found out to be 1 m 
10 frames. Thus the computational load reduction would be by a factor of lO- Thus toe 
extraction of color information would be started by processing the 1 fi:ame foUowed by 
the 10* frame and so on. 
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The average color information in each frame for each region of the channel is extracted 
by summing up all the pixels in that region and dividing by the total number of pixels m 
that region for each channel. The equation for the extraction of the average color 
information for each region for one channel is shown below: 



Rred 



nxm 



If the region under consideration is Rl, then My of size 360x100 with n equal to 360 
and m equal to 100. The above equation gives us the average of all the pixels for the red 
channel. Thus the average color for particular region would now be a triplet, 

Rm ~ \R,^d'Rffeen'Rblm\ 

The same procedure is repeated for all the regions and for all the channels within each 
region. 

Next in orda: to set the lights, a mapping transformation needs to be performed between 
the TV and light units. This is achieved via a standard set of equations that take as input 
the measured color primaries from each LED unit. The color priiharies for the red, green, 
blue and the reference white color components are acquired by usmg a color 
spectrometer. Once the primaries are obtained, the transformation process is as follows: 

(a) Given a set of chromaticity (red, green and blue primaries) co-ordinates and 
the reference white, compute the transformation matrix for mapping the 
average color information onto the XYZ color gamut space for both the FLAT 
TV as well as the LED units. This gives us two sets of equations : 

[X; Y; Z]=M, * [R;G;B] for Flat TV 

[X;Y;Z]=M2*[R';G'B'] forLED's 

(b) The mapped RGB values for the li^t units could be found by solvmg the 
following: 

[R';G';B'] = Uz'' * M, * [R;G;B] 

In steps (a) above, [R; G; B] corresponds to the triplet which is nothing but the computed 
average color information for a particular region for all channels. The general method for 
computing the matrix M is shown below: 

Given the chromaticity coordmates of an RGB system (Xr,yr), (Xg,yg) and (Xb,yb) andtiie 
white point (Xw,yw), the method to compute the 3 x 3 matrix for convertmg RGB to XYZ 
is as follows: 

[X.YZ\ = [RGB][M\ 
where 
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The above method is used for obtaining Mi and Ma and [R' G' B'] by following step^) 
above Thus [R';G';B'] is the transformed color information for a particular region. Ihe 
same process is repeated for obtaining [R' G' B'] for each of the 6 regions. 

Once the color information has been extracted and transformed for the Ij' and the 10* 
frame, then the color mformation for intermediate frames is computed by ^^^t computmg 
the absolute difference between the 1^' and the 10*'' frames divided by 0 Jer^n ^e 
called as fractional difference. Thus tiie 2"^ frames color ui^nnation wodd be the co or 
information of the 1** frame plus the fractional difference. Similarly the 3 frames color 
mformation would be the 2""^ frames color information plus the fractional difference and 
soon. 

The color information is then sent to the light units so that they can be triggered. It is also 
important to note that the width of each region could be vaned and the number of regions 
m the frame could also be varied. As an example, instead of usmg 6 regions, one could 
use only 4 regions as well. In such a case, the thresholded color mformation for &e upper 
Son could be sent to both the LED panels located at the top of the TV. Furthermore, 
the solution discussed above could be realized in software or via a programmable 
hardware platform such as EPLD, etc. 



